The peer srx should return entries with the FI_MSG/FI_TAGGED and
FI_RECV flags set

The msg_size field in the peer_rx_entry needs to be set in the
expected path to the number of bytes allowed to be copied.
This is either the size of the message (from the attr->msg_size
paramter) or, if the buffer is not large enough to hold the entire
message, the size of the buffer.

This also fixes setting the message size and flag fields on the unexpected
multi receive path. This case is a bit different because it not only has to
account for the message size and buffer size, but also for the owner entry's message
size and flags

Signed-off-by: Alexia Ingerson <[email protected]>

The util srx/peer srx should return the entry with the op flags
(FI_MSG/FI_TAGGED + FI_RECV) already set. Remove from the shm code

Signed-off-by: Alexia Ingerson <[email protected]>

ubertest implementation currently requires FI_RMA_EVENT when using RMA and counters
This will cause tcp to return ENODATA for these combinations and cause runfabtests
to fail.
This should get updated in ubertest to not require it but remove testing for now

Signed-off-by: Alexia Ingerson <[email protected]>

The rxm SAR segment type enum was defined inside another struct.
While techincally ok, this made it difficult for editors to find
the type and reported compiler errors. This cleans it up to make
it more readible and easier for editors to find the type

Signed-off-by: Alexia Ingerson <[email protected]>

Add application side support for FI_AV_USER_ID which requires
saving the fi_addr input as the internal fi_addr (for both the
peer API srx use case and for reporting unique source address
information).
When supporting the capability for the application, remove it
form the core provider information as it is only required on the
top layer

Signed-off-by: Alexia Ingerson <[email protected]>

Support using the peer APIs by default using the util peer helper
functions. Instead of going through the rxm-specific functions to write
to CQs and counters, use the ofi_peer_cq/cntr APIs which use the
owner ops. In the default case where rxm is not being used as a peer
these will go to the regular ofi_cq_write functions.

Signed-off-by: Alexia Ingerson <[email protected]>

Remove rxm implementation of receive queues and leverage the util
srx implementation which supports the peer srx API. This allows rxm
to use the peer API calls to match receives.
To do this, move the rxm protocol information from the receive entry
into the rx_buf and allocate it dynamically as needed to track protocol
information. This allows rxm to use the default peer_rx_entry instead of
its own custom receive entry.

With this last piece of the peer API implemented, rxm can also now
advertise full support of the FI_PEER capability. Just like the
FI_AV_USER_ID capability, rxm removes the bit from the core provider
info as it is only a requirement from the application side and not
from the message provider

Signed-off-by: Alexia Ingerson <[email protected]>